This invention is related to systems and methods for stationary object detection, including driver situation awareness, vehicle collision warning, collision avoidance and/or adaptive cruise control systems as examples. More particularly, this invention relates to improvements in stationary object detection, and accurately determining the likelihood that a objects detected using an object sensing system are not normally present at a sensed location.
Object detection systems can be useful in a variety of environments, including for use in land-based mobile machines and vehicles where there is a threat of collision with the objects. Such land-based mobile machines or vehicles may include subterranean or open-pit mining machines, on-road or off-road vehicles, robotic machines or the like. In the varying environments in which such machines are used, it should be evident that various obstacles and objects are within the possible path of the machine, and wish to be detected and often avoided. For example, vehicles may encounter conditions where there is no visibility of objects within the possible path of the vehicle. Collision warning systems are currently being developed to provide a warning to the commercial and/or passenger vehicle driver whenever an onboard sensor detects that an object presents a potential for a collision. For safety concerns, these systems must use a low threshold in determining whether the identified target object is a potential collision hazard in order to prevent the system from missing a bona fide hazard target. This results in the collision warning system sometimes generating a collision alert in response to non-hazard targets and/or clutter such as xe2x80x9croad furniturexe2x80x9d. Typical non-hazard targets include road signs, bridges, fences, guard rails and road-side burms, etc. These xe2x80x9cfalsexe2x80x9d alerts, also known as xe2x80x9cnuisance alertsxe2x80x9d, are an annoyance and can happen so often that the driver may delay reacting or ignore the warning and not react to a bona fide warning. Poor discrimination of such non-hazards from true hazards limits the effectiveness and reliability of such collision warning systems.
There are currently several types of vehicle-based collision warning and adaptive cruise control products for use in commercial and passenger vehicles. The first system is known as Adaptive Cruise Control (ACC). ACC is an enhancement to conventional cruise control which has the ability to detect objects ahead of the moving vehicle when cruise control is engaged to improve the xe2x80x9ccomfort and conveniencexe2x80x9d of the system. It is not employed as a safety system as the driver must still pay attention and respond to potential hazards, and is often marketed as simply a xe2x80x9cmore convenientxe2x80x9d form of cruise control, even though some evidence exists that its use offers safety benefits, as well, compared with conventional cruise control. When the ACC host vehicle is cruising at its cruise-control set-speed, the ACC automatically slows down the host vehicle to match the speed of a somewhat-slower-moving vehicle traveling in its lane ahead and establishes an xe2x80x9cappropriatexe2x80x9d following distance, which in many ACC designs the driver has a role in setting, within defined minimum and maximum distance limits. After that, the ACC enters xe2x80x9cdistance-controlxe2x80x9d mode, matching the forward-vehicle""s speed (unless it accelerates beyond the host vehicle""s set-speed), as long as it is present. If another vehicle cuts in and stays in the lane, the ACC will decelerate the host vehicle to re-establish the xe2x80x9cappropriatexe2x80x9d distance and then re-enter xe2x80x9cdistance-controlxe2x80x9d mode. Whenever the lane ahead clears, due to either vehicle changing lanes, the ACC will cause the host vehicle to smoothly accelerate up to its cruise-control set-speed and keep that speed until a slower-moving vehicle is again detected in its lane ahead. The ACC typically uses a stand-alone forward-looking sensor system, which can not accurately judge whether a detected object is in fact on the roadway in the host vehicle""s lane without generating an unacceptable number of nuisance alerts. Accordingly, it is generally understood in the automotive industry that ACC has been designed not to address this situation, and some ACC systems have even been designed not to respond to moving vehicles in their lanes traveling at speeds slow enough so that the driver might not realize they are moving. The reason for this is to attempt to avoid giving the driver a misimpression that the ACC does or should respond to stopped vehicles, with the result being that such an ACC provides even less functionality than it is capable of providing, due to this human-factors consideration. With an improved ability to sense objects and discriminate stationary objects, an improved version of ACC could developed to provide additional capabilities, such as the ability to respond to all vehicles in the host vehicle""s lane, whether moving at normal highway speeds, moving slowly, moving and coming to a stop, or even stationary when first detected.
Another product is known as the Forward Collision Warning System (F-CWS). The F-CWS operates as an advisory system to the driver, who operates the vehicle in the normal way and is still fully responsible for safe operation. Various warnings are given for a wide range of potentially-dangerous situations involving moving vehicles and stationary vehicles or other large objects. Unlike ACC, the F-CWS is designed to operate while the vehicle is under full control of the driver, rather than while cruise control is operating, and also to warn the driver of a wide range of potentially dangerous situations, including those involving stationary objects that are judged by the F-CWS to lie in the expected path of the host vehicle. Due to the difficulty of distinguishing stopped vehicles or other stationary objects in the path of the vehicle, when compared to normal roadside or overhead highway structures, current products of this type on the sometimes warns the driver of a potential collision when there is no hazard, resulting in a nuisance alert. This is widely believed to be a primary reason why such products were first introduced on commercial vehicles, and have not been introduced on passenger vehicles, even in markets where the commercial vehicle products are currently in use (including the United States). Although an occasional nuisance alert can be an annoyance to a professional driver operating a commercial vehicle, such drivers are generally provided specific training on the use and limitations of these products, and with their extensive driving experience, often simply regard the occasional nuisance alert as a minor distraction. However, there is wide concern that passenger vehicle drivers, who may have much less driving experience and who cannot generally be required to undertake product-related training similar to commercial drivers, may react unpredictably to any nuisance alerts generated in error by a safety-related product such as F-CWS. Various approaches have been used in attempts to minimize these nuisance alerts. One such approach involves delaying an eventual warning while more information is taken to evaluate the probable location of a detected stationary object and the expected location of the host vehicle""s lane ahead. If the stationary object does not seem to be in the projected host vehicle""s lane, it is usually judged not dangerous. On the other hand, if the object is confirmed as a hazard, the warning will be delayed. This will allow the driver less response time to avoid the collision with the object. At the same time, the viewing angle of the sensor system may be purposely limited to help reduce nuisance alerts, which may limit the effective capabilities of the system.
Although attempts have been made at improving the sensor to better identify objects, such sensors have not resolved the problems to date or are too expensive for incorporation in commercial or passenger vehicles. Infrared laser sensors have been developed which provide both horizontal and vertical scanning, and which give detailed object sensing resolution; however, such sensors to date have had difficulties operating acceptably in foul weather and with respect to objects whose reflectivity may be significantly reduced by ice, snow, or even road grime. Radar technology may overcome much of these environmental problems; however such radar products that provide both horizontal and vertical scanning require so much hardware and software to operate that they remain much too expensive for commercial and passenger vehicle use. Current radar-based commercial or passenger vehicle products can provide limited information about detected objects: distance, relative speed, and horizontal position (sometimes including estimated horizontal width). None of them offer information about the vertical position of a detected object, which can lead to uncertainty at all but very close range whether an in-path detected object is actually in the vehicle""s path or is simply an overhead structure. Other attempts to improve the current F-CWS technology utilizing stand-alone sensors have not resolved the production of nuisance alerts, particularly in locations where sharp curves, low bridges, tunnels, and/or hills cause the system to mistakenly regard the detected stationary objects to lie on the roadway in the host vehicle""s path. The present state of the art for vehicle use does not have the ability to reliably detect whether a detected stationary object is at road level or is perhaps simply an overhead object such as a low bridge or tunnel opening. The ability to precisely locate a stationary object detected ahead to be at the horizontal coordinates matching the host vehicle""s lane is not adequate to determine whether the object is in the roadway or overhead.
The only existing forward-looking collision warning system in commercial use in the United States utilizes Doppler-based modulation for its radar sensor. Doppler processing is based on identifying objects by detecting the change in frequency between an emitted signal and a return signal reflected back from the detected object, and this xe2x80x9cDoppler shiftxe2x80x9d relates directly to the detected object""s speed relative to the sensor""s transmitting antenna. With modern digital radar signal processing, typically using fast Fourier transform (FFT) methods, such radar sensors are designed to separate all detected objects into a number of separate xe2x80x9cDoppler binxe2x80x9d categories, based on their Doppler frequency shift, and thus speeds relative to the sensor. The design of typical sensors used for vehicle-based ACC and CWS typically can identify an object separately from others if its relative speed is only a few tenths of a mile per hour different from the rest. In practice, this has proven very effective overall for these systems, but one result is that, with some exceptions for objects nearer the edges of the radar beam, all stationary objects detected within the radar beam appear to be moving at the same relative speed (they are fixed and the vehicle is moving towards them at some speed). Such Doppler-based radars are extremely accurate with respect to relative speed, as well as for distance (and horizontal angle, depending on the antenna design) to individual objects. However, for stationary objects, the distance and horizontal angle (if provided) are calculated for all objects whose relative speeds place them in the same Doppler bin, and for stationary objects, this results in distinct objects not being recognized as such, and the apparent distance and horizontal angle to a significant stationary object can be strongly influenced by other stationary objects within the radar beam. Because the radar beam is typically no more than 12 degrees wide and 4 degrees high, a fairly small amount of the stationary environment influences the results, but since the xe2x80x9creflectivityxe2x80x9d or radar cross section (RCS) of an object and its nearness to the sensor are strong influences, the xe2x80x9capparentxe2x80x9d location of an individual object when first detected is not only not precisely accurate with its physical location (because the result is being influenced by other stationary objects), its xe2x80x9capparent locationxe2x80x9d changes, often moving closer to the sensor, as the sensor approaches the object. This is due to the fact that the individual object takes up more of the radar beam as it is approached, so it has greater influence than at greater distance. Further, the distance itself is a strong influence on the radar return, so as it is approached, its smaller distance also gives it more influence on the calculations for the location of it as well as all other objects in its Doppler bin. The net effect is that individual stationary objects can only be precisely located when they are fairly close to the Doppler-based radar sensor, and at varying distances, their xe2x80x9capparent locationsxe2x80x9d appear to move in space as they are approached.
Another potential product is the Short-range Collision Warning System (S/R-CWS). At present, a convenience-oriented product generally called Parking Aid is available on vehicles, which is a limited-function parking assistance system, generally utilizing ultrasonic sensors to warn the driver when the host vehicle is getting too close to another object in the rear or front while parking. These systems only need to operate over very short distances and for very slow speeds. Most vehicles have significant blind spots immediately to the rear, and many commercial vehicles have a blind spot immediately ahead, so a S/R-CWS designed to alert the driver to any dangerous situation, especially while the vehicle is moving in reverse, must meet difficult design requirements. It must reliably and quickly detect people, especially small children, who may be playing or otherwise be behind the vehicle without the driver""s knowledge. It must provide warnings to the driver for such detections with adequate time for the driver to respond, while the vehicle is moving at normal speeds for maneuvering in parking or work areas, or backing down a driveway. To do so, and to reduce the potential for missed detections to an absolute minimum for safety reasons, it is difficult to design such stand-alone systems with current technology without having a significant potential for nuisance alerts. To be an acceptable product, it must have very few nuisance alerts, or else risk having the driver ignore critical alerts in areas where nuisance alerts might occur.
Although not discussed in detail, there are other collision warning systems which are also susceptible to nuisance alerts. Conquering the problems associated with nuisance alerts in such systems will not only allow greater deployment and acceptance of the ACC, F-CWS and the S/R-CWS and other related systems, but will also allow for the introduction of advanced collision avoidance systems. The collision avoidance systems have the ability to not only warn the driver, but will also take action to help avoid the collision such as slowing or stopping the vehicle. These systems have the potential to reduce collisions and improve driver safety, beyond the improvements possible with ACC, F-CWS, and S/R-CWS products. Additionally, in a variety of other environments, such as in the use of mobile machines such as robotic vehicles or the like, reliable object detection is essential, and similarly, the accurate measure of whether a sensed object is normally present at a sensed location will facilitate operation of such machines, in some cases to avoid the object but in other cases to investigate it.
Therefore, there remains a need in the art for a method and apparatus for accurately measuring the likelihood that detected stationary objects are normally present in an environment at a remotely sensed location. Such improved measuring capabilities would in turn improve suppression of nuisance alerts in such systems as adaptive cruise control and/or collision warning and/or collision avoidance systems. Such improved measurement would also enable sensing systems to operate at their fullest effective range and ability.
To overcome the above indicated problems, the present invention provides systems and methods for determining the likelihood that stationary objects detected by a sensor in a geographic area are unusual, based on comparing remotely-sensed characteristics of those objects with characteristics detected by similar sensors operating with locating devices in the same geographical area on previous occasions. This invention is equally applicable to all known sensor technologies, including Doppler-based radar, because the characteristics of detected stationary objects, including apparent relative location, detected from a particular vehicle location are compared with characteristics detected on previous occasions by similar technology sensors operating at the same vehicle location; as a result, xe2x80x9capparent locationxe2x80x9d of stationary objects calculated from a vehicle location is compared with a similarly-calculated xe2x80x9capparent locationxe2x80x9d using detection by similar technology sensors from the same vehicle location. This approach has significant advantages, compared with comparisons between calculated absolute position based on xe2x80x9capparent locationxe2x80x9d calculated from Doppler-based sensor data and actual physical location recorded in a navigation-based map database, however detailed and accurate. In addition, the present invention offers advantages for non-Doppler-based sensor systems, due to the fact that it provides signal return comparisons, in addition to possible other characteristics detected on previous occasions by similar sensors, which give information in addition to location to help distinguish the presence of multiple stationary objects which appear to be at the location of an object which is normally there. This helps reduce the uncertainty of comparisons based solely on location-matching when location precision better than 1 meter is unavailable. Compared with the use of a navigation-based map database and similar sensors, the present invention also addresses the difficulty of reliably detecting whether a stationary object is at road level or overhead, by providing the ability to compare the signal return strength with what should be normal from each point in the approach path; in the case of a stationary vehicle beneath a low overhead structure, the relative locations will match the normal ones as the vehicle approaches, but the return signal magnitude should be greater than normal, due to the presence of the roadway-level object which is returning the signal in addition to the overhead structure which would normally be the only stationary object at that location returning the signal.
Such systems and methods may be used for minimizing nuisance alerts in onboard object detection systems such as collision warning, collision avoidance, and/or adaptive cruise control systems. The system includes at least one vehicle mounted sensor capable of sensing at least a stationary target object and providing data related to the target object. The system also comprises a locating device which is capable of determining and providing data related to the location of the machine or vehicle and a processing unit which receives the data from the sensor and the data from the locating device. The processing unit is configured to determine a probability estimate or measure of likelihood that the target object is not a normally present object or is otherwise unusual, based upon a comparison to previously recorded data from a reference storage device. The reference storage device stores the previously recorded data acquired from at least one similar sensor and a vehicle locating device while operating in the same geographical area where the target object is being detected, and/or data derived from such previously recorded data.
It is an object of the present invention to provide a system which will reliably evaluate whether a target stationary object is unusual based on its remotely-detectable characteristics, and is therefore a bona fide object, such as a potential collision object, at a distance up to the operating distance limit of the sensor.
It is another object of the invention to provide such evaluation of a target stationary object to a collision warning system to improve its decisions whether and how to alert the driver, and to an avoidance or ACC system to improve its decisions whether and how to alert the driver and how to control the vehicle. In this manner, the invention assists the identification of bona fide hazard targets and the suppression of collision warnings produced in response to various non-hazard targets and/or clutter commonly encountered in the operating environment of many land-based vehicles.
It is another object of the invention to provide a vehicle""s driver or a mobile machine""s remote operator with improved situation awareness of unusual stationary objects, to allow the driver or operator to begin evaluating the situation, possibly in advance of alerts or other actions initiated by collision warning, ACC, or collision avoidance systems. This is normally done by identifying to the driver or remote operator the location of detected stationary objects which are not normally present or which for other reasons are sufficiently unusual. Since a collision warning, ACC, or collision avoidance system may require more time to sufficiently evaluate whether the unusual stationary object may also be hazardous, it is very possible that the driver will receive such situation awareness information before another system provides alerts or takes any other type of action. If the driver or operator agrees that objects identified by the situation awareness information do appear to be unusual, even if not dangerous, then such unusual but non-hazardous object identifications should not be considered xe2x80x9cnuisance alertsxe2x80x9d, and should not reduce the effectiveness of the invention. An example of such a situation is a car parked in an emergency lane adjacent to the host vehicle""s lane, but too far away for a collision warning or collision avoidance system to determine with acceptable confidence whether it is in the host vehicle""s lane. If called to the driver""s attention as an unusual stationary object, the driver should quickly realize that there is no little to no danger, but may still appreciate the situation awareness message, so the driver or operator could begin changing lanes or simply watch more closely in case someone is near the car who might not see the approaching host vehicle and step into the lane. Such situation awareness information would normally be expected to work in conjunction with an associated collision warning, ACC, or collision avoidance system operating, but in some cases may operate as a stand-alone capability without full collision warning, ACC, or collision avoidance systems actively engaged or even present.